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The Propulsion IVHM Technology Experiment (PITEX) has been an on-going research effort 
conducted over several years. PITEX has developed and applied a model-based diagnostic system for the 
main propulsion system of the X-34 reusable launch vehicle, a space-launch technology demonstrator. 
The application was simulation-based using detailed models of the propulsion subsystem to generate 
nominal and failure scenarios during captive carry, which is the most safety-critical portion of the X-34 
flight. Since no system-level testing of the X-34 Main Propulsion System (MPS) was performed, these 
simulated data were used to verify and validate the software system. Advanced diagnostic and signal 
processing algorithms were developed and tested in real-time on flight-like hardware. In an attempt to 
expose potential performance problems, these PITEX algorithms were subject to numerous real-world 
effects in the simulated data including noise, sensor resolution, command/valve talkback information, and 
nominal build variations. The current research has demonstrated the potential benefits of model-based 
diagnostics, defined the performance metrics required to evaluate the diagnostic system, and studied the 
impact of real-world challenges encountered when monitoring propulsion subsystems. 


I. Introduction 

For several years researchers at NASA’s Glenn Research Center, Ames Research Center, and 
Kennedy Space Center have been engaged in a cooperative effort to develop and demonstrate an 
advanced diagnostic capability for a space-based propulsion application. This effort, called the Propulsion 
Integrated Vehicle Health Management (IVHM) Technology Experiment (PITEX), was a key element of 
the Northrop Grumman Corporation Space Launch Initiative Risk Reduction Task, and the cornerstone of 
their IVHM Vehicle Test Bed demonstration at the Jet Propulsion Laboratory. 1 The PITEX research was 
based on legacy work from the NASA IVHM Technology Experiment for X-Vehicles (NITEX), funded 
through the Future-X Program Office. NITEX was selected to fly on the X-34 Reusable Launch Vehicle 
(RLV) developed by Orbital Sciences Corporation (OSC). The intent of this research was to extend 
IVHM technologies by applying them to pertinent RLV subsystems. The NITEX research initially 
focused on the X-34 Main Propulsion System (MPS) during all phases of operation. 2,3 The research was 
scaled back to address X-34 MPS subsystems active during Captive Carry because, during this phase, 
critical failures would endanger humans involved in the vehicle testing. 

After the X-34 program was cancelled, PITEX carried forward the previous research by building 
upon a prototype diagnostic system that was developed during the NITEX period. The PITEX research 
focused on the continued maturation of diagnostic technologies as applied to subsystems relevant to 2nd 
Generation RLVs and in particular, the X-34 MPS. These diagnostic technologies were aimed at real-time 
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safety applications, and therefore the PITEX demonstration required the establishment of metrics to 
assess the real-time diagnostic performance in terms of accuracy, speed, and computer resources. 
Subsequent improvements to the PITEX software have been quantified by these metrics and reported 
previously. 4,5 

For any diagnostic system, demonstration and validation are difficult to perform, because in order to 
determine that the diagnostic system is functioning properly, the monitored system must fail. Ideally, 
testing of the diagnostic system would occur directly on the hardware it was designed to monitor. 
However, the costs and risks associated with these types of failure tests are prohibitive, especially for 
rocket propulsion systems. To overcome this problem, simulation-based testing and validation of 
diagnostic systems must become an acceptable process. Due to the early cancellation of the X-34 
program, PITEX development and testing relied solely on simulated data and exposed the importance of 
simulation-based testing and the level of realism that testing must provide. 

Real-world challenges encountered in the development of PITEX and how they were managed are the 
focus of this paper. Primarily, the lack of experimental data for development, testing, and validating the 
diagnostic system and its robustness to system uncertainties and variations are addressed. In addition, 
improvements to the development process for the diagnostic model are also identified. This paper is 
organized as follows. A description of the PITEX real-time demonstration is provided next. The software 
and hardware used, as well as the domain application and the test scenarios, are defined. After that, 
realistic sensor and hardware challenges encountered in developing the PITEX diagnostic system are 
reviewed, and recent enhancements implemented to address these challenges are presented. Finally, 
impacts on the PITEX demonstration test results, as a byproduct of incorporating these changes, are 
reported and discussed in the context of validating the diagnostic solution and the management of 
computer resources. 


II. PITEX Demonstration Overview 

The PITEX demonstration system architecture is based on the overall NITEX flight experiment 
architecture and is designed to test a subset of software components from the original architecture. The 
demonstration system features simulated propulsion data being processed by diagnostic software on the 
“flight-like” Real-Time Ground Unit (RGU) and display software hosted on a Ground Processing Unit 
(GPU). 


A. Hardware Specifications 

The PITEX diagnostic software resides on a Radstone PPC4A-750 VME single Board Computer. The 
card is housed in a chassis with a VME backplane and a SCSI hard drive. I/O ports provide both serial 
and Ethernet accessibility to the card. The PITEX diagnostic software is compiled in the Tornado II 
VxWorks (release 5.4) environment using the C and C++ compilers supplied with VxWorks. The 
VxWorks kernel is based on the Radstone board support package, release 1.2/1. 

The GPU hardware is a Pentium III based computer, with the clock speed of 550 MHz, 256 MB of 
RAM, and 30 GB of hard disk storage. The GPU software has been successfully compiled and run using 
the gee compiler on Redhat Linux versions 6.2, 7.3, and 9.0. The GPU communicates with the RGU 
through a TCP/IP Ethernet connection. 
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B. Software Specifications 


The software components of the PITEX demonstration system consist of an MPS simulator, 
Telemetry Input System (TIS), Monitor software, Real-Time Interface (RTI), Livingstone inference 
engine 6 (which uses a compiled qualitative model of the MPS), Results Output System (ROS), and the 
GPU. The overall architecture is shown in figure 1 . The virtual propulsion system simulates the flight-like 
sensor data associated with the particular mission phases within the nominal and failure scenarios. These 
data are stored in binary files prior to diagnostic system testing. The TIS reads these data sets and 
incrementally stores the information so as to provide access to the modules in the same manner and time 
frame as what was to be experienced by the system during the real-time operations on an actual test flight. 
After the TIS stores the data on-line to simulate a data sweep, the data are accessed by the Monitor 
software where pertinent features of the propulsion system are extracted and transformed into qualitative 
information. This information is then passed through the RTI to Livingstone, where system-level 
diagnostics are performed using a high-level qualitative model. The diagnostic output is collected by the 
ROS and sent to the GPU for display. Each of these software modules is described in greater detail in 
reference 4. 



X-34 Virtual 
Propulsion 
System 



Diagnosis Requests 


Flight 
Software 
on PPC4A-750 



Ground Processing Unit 


Figure 1. — PITEX Demonstration Architecture. 


C. Domain Application 

The PITEX development and testing has focused on the X-34 MPS during the Captive Carry phase of 
operation. The portion of the X-34 MPS modeled for PITEX was restricted to the liquid oxygen (LOX) 
and standard kerosene rocket fuel (RP-1) propellant feed circuits and the supporting pneumatic and 
hydraulic systems. The Captive Carry phase is the flight period where the X-34 vehicle, while attached to 
an L1011 aircraft, departs from the tarmac and ascends to 38,000 feet where it is released for flight. 
During this period of time, which is approximately 2.5 hours, the MPS maintains the cryogenic LOX 
propellant in a stable state through a conditioning process. Then it prepares the engine for flight, 
introducing LOX and RP-1 propellant into the feed lines. A timeline of these events is presented in 
figure 2. 
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Figure 2. — X-34 Captive Carry Design Reference Mission. 


PITEX was developed to detect and isolate certain fault modes and utilized 25 distinct scenarios to 
demonstrate its diagnostic capabilities. Twenty-four of these scenarios were selected based on the Failure 
Modes and Effects Analysis of the X-34 MPS, input from domain experts, and the feasibility of simulating 
the failure on the ground. Brief descriptions of the scenarios and their corresponding fault injection times 
(associated with the timeline shown in figure 2) are listed in table 1. Figure 3 is a schematic of the 
X-34 MPS that shows the location of the components along with the sensors available for diagnostics. As 
noted in table 1, for some components more than one failure mode is considered, and there can be some 
confusion about the failure descriptions. For instance, there is a difference between a valve sticking 
closed and failing closed. In the former case, the valve “sticks” closed when it has been commanded to 
open but remains closed. In the latter case, the valve “fails” closed when it should remain open but closes 
without command. Analogous descriptions apply to valves sticking open and failing open. 


TABLE 1.— PITEX DIAGNOSTIC DEMONSTRATION SCENARIOS 



Description 

Fault 

Injection 

Time 

(seconds) 


Nominal data set 

N/A 

1 

LOX Feed Valve, PV03, sticks closed 

9410 

2 

LOX Feed Servo Valve, SV33, fails closed 

9539.79 

3 

LOX Tank Vent Relief Servo Valve, SV3 1, sticks open 

2706.9 

4 

LOX Tank Vent Relief Servo Valve, SV3 1 , fails open 

5000 

5 

LOX Tank Vent Relief Valve, VR01, sticks open 

2706.9 

6 

LOX Tank Vent Relief Valve, VR01, fails open 

5000 

7 

LOX Tank Vent Relief Servo Valve, SV3 1 , sticks closed 

5167.4 

8 

LOX Tank Vent Relief Servo Valve, SV3 1, fails closed 

3331 

9 

LOX Tank Vent Relief Valve, VR01, sticks closed 

5167.4 

10 

Pressurization Regulator, RG1 1, regulates high 

9000 

11 

Pressurization Regulator, RG1 1, regulates low 

9642.67 

12 

LOX Tank Pressurization Valve, SV03, sticks open 

9735.29 

13 

LOX Tank Pressurization Valve, SV03, sticks closed 

9766.05 

14 

The open microswitch, MMSW205X, fails on the LOX vent/relief solenoid valve, SV31 

3301.7 and 
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Description 

Fault 

Injection 

Time 

(seconds) 



7260.8 

15 

The close microswitch, MMSW213X, fails on the LOX feed valve, PV03 

9410 

16 

The RP-1 feed valve, PV02, fails closed after the RP-1 bleed has been initiated 

9359 

17 

The RP-1 vent/relief pneumatic valve, VR06, fails open 

9379 

18 

The primary RP-1 tank pressurization valve, SV02, sticks closed 

9383.86 

19 

The primary RP-1 tank pressurization valve, SV02, sticks open 

9384.71 

20 

The open microswitch, MMSW205X, fails on SV31, after that, SV31 fails closed 

3301.7 

21 

GHe pressurization system pressure regulators, RG1 1 and RG01, both regulate high 

9000 

22 

Two of the LOX vent line pressure sensors, MPRE202P and MPRE212P, fail high 

0.0 

23 

Two of the LOX vent line pressure sensors, MPRE202P and MPRE212P, fail low 

0.0 

24 

The last lower flapper valve in the aft LOX tank fails shut 

9199 


MRTD203T 


MMSW303X 



MRTD107T 


1 MPRE103P 


Figure 3. — PITEX X-34 Main Propulsion System Schematic. 


III. PITEX Development Challenges 

A key challenge that any diagnostic system must face is the ability to handle real-world uncertainty 
and variances that naturally occur, while at the same time providing sensitivity with respect to detecting 
and recognizing anomalous conditions. Consequently, one of the key activities of the PITEX research was 
to expose the diagnostic software to various real-world effects. The purpose of this was to determine 
whether the current system was robust to these effects and if not, how those effects could be addressed 
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within the diagnostic solution. Another goal was to lay the groundwork for a well-designed diagnostic 
modeling environment. The objective of this is to reduce the life cycle costs (LCC) (costs of development, 
maintenance, testing, and validation) of the diagnostic solution. The remainder of this section will 
describe the real-world effects (sensor and hardware uncertainties) and Livingstone diagnostic modeling 
changes that were addressed during the development of PITEX. 


A. Sensor Uncertainties 

One of the primary difficulties in fault detection and diagnosis is the ability of the diagnostic software 
to discern what is actually occurring in the system from the sensor information. The sensor signal 
contains a varying level of noise that is unrelated to the physical state of the monitored system and is 
superimposed upon the actual signal. The diagnostic system must be conservative in its determination of 
the actual sensor signal in order to ensure that the observation is truly a representation of the physical 
system and not merely a product of sensor noise or sensor failure. At the same time, the diagnostic system 
must be sensitive enough to provide a timely assessment of the system health that will enable remediation 
in the event of a fault condition. 

Another important sensor information consideration is the analog-to-digital (A/D) resolution. The 
A/D resolution is based on the amount of digital information retained for each sensor and establishes the 
ability to distinguish system events from the sensor signal. The resolution is a function of the dynamic 
range and the digital word size allocated to that sensor during the A/D conversion process. The resolution 
defines the smallest voltage change that can be measured by the AD converter; as the A/D resolution 
decreases (the resolution value becomes larger), the ability of the sensor signal to be used to distinguish 
changes decreases. Each sensor may have a unique resolution associated with it. This complicates the 
problem when a diagnostic algorithm is attempting to cross-correlate events in related sensors that have 
different resolutions. 


B. Hardware Uncertainties 

The performance of a system is based upon the interactions of the subcomponents and the external 
boundary conditions. There are variations that manifest themselves in the data that are not indicative of a 
system fault, but they are caused by hardware component differences or changes in the system’s boundary 
conditions. Some of these uncertainties include: variations in the actuation time for the opening or closing 
of valves, regulator set points, initial conditions for the propellant tank, boundary conditions for the dump 
lines, and the atmospheric conditions. These types of variations are considered normal and must be taken 
into account when establishing the threshold limits that the diagnostic system is using to detect actual 
system events. Furthermore, the diagnostic system must demonstrate the capability to process these 
nominal system variations without any false alarms. 

Time delays in the system can also be attributed to hardware uncertainties. In mechanical or fluid 
systems, more so than an electrical system, there are inherent delays in physical properties to system 
changes. The reaction times of electrical switches or indicators are virtually instantaneous. Moving 
mechanical components however, such as a valve, often require tens to hundreds of milliseconds to 
completely open or close. In addition, there are physical delays in the system that also must be taken into 
account. For a fluid system there is a period of time that the system will be in an unsteady state. The 
length of this transient period is different depending on the initial state and disturbances to the system and 
is further complicated by the presence of any faults. Correlating all of these variations in real time can be 
a difficult process. The diagnostic system must handle all these types of delays, ensuring that all pertinent 
information is available before an analysis on the observations is performed, while at the same time 
providing a responsive and timely analysis. 
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C. Diagnostic Modeling Environment 


The ease at which a diagnostic system can be developed is important when the goal is to reduce the 
development time and overall LCC for a new application. Toward that end, being able to reuse 
component modules for previously developed and verified systems is an important capability to develop. 
Presently, each component in the Livingstone model must be defined individually, and this can be 
cumbersome and time consuming for a complicated system such as the X-34 MPS. For instance, similar 
components, such as solenoid valves, have different models depending on their placement in the system. 
To simplify this process, a more robust diagnostic modeling environment is required. One method is to 
use a single valve component type that has the same basic underlying logic and thus is generic to 
implement. Modifying the diagnostic model to use these generic representations instead of specialized 
components for each instantiation would save development and testing time when creating the diagnostic 
model. As another benefit, the Livingstone development environment becomes a library of components 
that can be utilized for other applications and facilitates interoperability with other diagnostic systems. 


IV. PITEX Developments and Enhancements 

This section will outline the modifications that were implemented in PITEX as part of that research 
effort to address the issues of sensor and hardware uncertainties. In addition, an overview of the generic 
component development for the Livingstone diagnostic modeling environment is also presented. 


A. Modifications to Address Sensor Uncertainties 

Initial versions of the PITEX software used thresholds to divide the response space of a sensor signal 
or processed signal into distinct qualitative regions. These regions were predefined based on anticipated 
system responses to various conditions. The PITEX Monitor software assessed the sensor signal to 
determine where it lies in the response space and maintained an array of recent qualitative assignments 
that were polled to determine what observations to report to the RTI. There were three problems with this 
method. 

1 . Potentially, narrow regions within the divided response space could be skipped and therefore not 
reported. 

2. The process was based upon predefined variables and was therefore sensitive to dynamic 
variations in sensor outputs such as noise levels. 

3. This technique was insensitive to slowly varying process changes and required large latency 
windows, on the order of 20 seconds, to compensate for the inability to detect and resolve the slow 
response. 

To solve these problems, a statistically-based algorithm was developed that would gather a population 
of the data, determine the mean and uncertainty of that population, and report all the regions that are 
overlapped by the population’s distribution. This version of PITEX addressed each of the problems 
previously mentioned by allowing the software to dynamically determine the uncertainty of the signal due 
to sensor noise and reflect that uncertainty into the observations passed on to the diagnostic model. The 
use of this approach reduced latency windows to approximately 6 seconds for the slow varying processes. 
Even with this modification, another issue still remained, that is the potential for “observation chatter” 
where the same sets of observations were constantly repeated. The chatter occurred when the sensor 
signal neared a threshold and the diagnostic system was required to respond to each set of observations. 
The frequent calls to perform diagnoses resulted in the consumption of valuable computer resources. To 
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resolve this issue, the policies within the RTI were modified to recognize this condition and eliminate 
repetitive diagnostic requests. 

The PITEX solution also addressed the issue of sensor resolution. For the X-34 MPS system there 
were several flow-field sensors that were correlated within the diagnostic system, but they were not at the 
same level of sensitivity or resolution. Since the essential information for determining a sensor’s 
resolution (i.e., memory storage and measurable range) is known a priori , a modification was made to the 
Monitor software to add the uncertainty due to sensor resolution to the observation, thereby enabling the 
cross-correlation of sensors with differing sensitivities. 


B. Modifications to Address Hardware Uncertainties 

Varying physical response times was an issue that was addressed by PITEX in order to reduce 
diagnostic times and increase diagnostic coverage. The primary constraint imposed by the Livingstone 
diagnostic engine was that it receives quasi-steady state information. Therefore, transient, dynamic 
information had to be filtered from the diagnostic process. Aside from the loss of key diagnostic 
information that may be contained within the transient period, this created a time delay where the PITEX 
system was forced to be dormant. These dormant periods, when the physical system is reaching steady 
state conditions, were defined as the physical latency periods. The times associated with these periods 
were determined empirically for various nominal system responses and constituted the majority of the 
overall diagnostic delays. In order to shorten the diagnostic delay, two key capabilities were added to the 
RTI which enhanced the ability of the PITEX to perform analysis during the transient periods. 

1 . The first was the ability to temporarily suspend only those observations directly impacted by the 
observed event or command. This allows the diagnostic system to continue to monitor unrelated 
areas of the system without increasing the chance of misdiagnosis by using an unsettled 
observation. 

2. The second was the assignment of pre-set latencies that are both sensor and event specific. This 
enables diagnostic analysis with only a subset of observations available. This may provide a faster 
diagnostic response, however it is tempered by the fact that the diagnosis is based on incomplete 
information. 

Additional work was conducted to expand the diagnostic coverage of Livingstone to transient 
regions. 5 In this new approach, a prototype of an event-specific transient monitor was designed and 
developed to provide an exact indication of when a transient event is completed. These changes may 
allow PITEX to provide diagnoses faster with complete available information and also potentially enable 
the inclusion of transient constraints to the Livingstone diagnostic models. These modifications to the 
reduced-prototype diagnostic model, demonstrated that Livingstone could provide diagnoses not only 
during steady sate, but also during the latency periods. 


C. Modifications to the Diagnostic Modeling Environment 

In order to reduce the development time of the diagnostic system, the modification of the diagnostic 
modeling environment focused on establishing generic modeling components. A new Livingstone model 
generation tool, called Oliver, was developed to utilize the generic components. As a demonstration of the 
potential application of generic component within Livingstone, the solenoid valves, sensors, and 
regulators of the X-34 MPS were selected as components for conversion to a generic format. For each 
component, model information was separated into two categories: that part specific to the X-34 MPS and 
the part generic to the component. The generic information was placed in a generic component module. In 
the future, when a specific module is needed, the generic component module can be extracted from a 
library and used as a framework to create the specific application. Examples of these new generic 
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components (solenoid valves, sensors, and regulators) were generated and incorporated into the generic- 
derived prototype PITEX model. 7 


V. Testing Results 

In general, the overall testing and demonstration of PITEX with the modifications that were described 
in the preceding section were conducted in two phases: (1) diagnostic validation and (2) computer 
resource utilization. 


A. Diagnostic Validation 

During the diagnostic validation phase, each scenario from table 1 was processed by the PITEX 
software and the resulting diagnostic output was analyzed. The potential fault candidates selected by 
PITEX were compared to the injected fault. Each candidate had to be justified and resolved based on the 
available commands and observations of the system. The following validation criteria were used in 
evaluating the diagnosis: 

• The injected fault was among the fault candidates reported. 

• All candidates in the diagnosis are explainable given the evidence available. 

In addition, the following acceptance criteria were used: 

• A diagnosis of a potential fault was obtained some time after the fault is injected. 

• Once the injected fault was diagnosed, it was in the list of candidates for the duration of 
the scenario. Note, that the injected fault may not have the highest rank in the list of 
candidates. 

Table 2 shows a sample of the diagnostic output from this validation testing for fault scenario number 
two. The fault injection time is the time in the simulation profile where the failure occurs. The diagnostic 
time is the initial time, following the injected fault, when the PITEX software provides diagnostic 
analysis output. With each diagnostic output PITEX provides a list of candidate solutions that could 
explain the available system observations. The highlighted candidate indicates the injected failure. The 
rank value is an internal metric within the diagnostic model that is generated from the order of magnitude 
of the probability of fault occurrence and would be the basis for diagnostic result selection. 


TABLE 2.— SAMPLE OF PITEX VALIDATION OUTPUT 


Fault Injection 
Time (seconds) 

Diagnostic Time 
(seconds) 

Candidate Solutions 

Rank 

9539.79 

9544.40 

LOX Feed Servo Valve, SV33, Stuck Closed 

1 

LOX Feed Valve, PV03, Stuck Closed 

1 

LOX Feed Servo Valve, SV33, Unknown Fault 

2 

LOX Feed Valve, PV03, Unknown Fault 

2 

LOX Feed Valve, PV03, Close MicroSwitch Faulty 
LOX Feed Valve, PV03, Open MicroSwitch Faulty 

3 

9550.0 

LOX Feed Servo Valve, SV33, Stuck Closed 

1 

LOX Feed Valve, PV03, Stuck Closed 

1 

LOX Feed Servo Valve, SV33, Unknown Fault 

2 

LOX Feed Valve, PV03, Unknown Fault 

2 


Identifies injected fault 


It is important to note that the PITEX diagnostic solution does not always generate a single candidate 
solution. For a set of observations, there will always be a list of possible solutions. This list of solutions, 
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called the ambiguity set, can be prioritized based on criticality and probability of occurrence. In the case 
where a specific failure needs to be distinguished from the remaining set, because the mitigating 
responses are drastically different, this would indicate or highlight where additional system information is 
required. The additional information may be in the form of additional sensors or more advanced 
processing of the current sensor suite. Also note, for the example fault scenario in table 2, the original 
candidate set evolves slightly in time. This is because as system observations become more refined in 
time, the diagnostic solution will become more focused. 

For the basic PITEX testing, all of the data sets that represent the fault scenarios from table 1 passed 
the validation testing as defined in the beginning of this section. In addition, in the list of candidate 
solutions for most of the failure scenarios, the top-ranked fault is the injected fault. The only exceptions to 
this are for fault scenarios 21 and 24. In fault scenario 21, two regulators in the pressurization system fail 
high. Because the likelihood of a double regulator fault is less than the pressurization sensor in that same 
line failing, the diagnostic system ranks the latter as more likely to occur. Likewise for fault scenario 24, 
the lower flapper valve in the aft LOX tank is not the most probable fault, and based on the diagnostic 
information there is no way to determine that it should be the highest ranked fault. 

Diagnostic timing is defined as the time interval beginning at the time the fault can be sensed to the 
time the diagnosis is reported. The sensed time is not necessarily the time of the fault. For example, the 
fault may be that a valve is stuck in the open position, but the failure would not be observable until the 
next time the valve is commanded closed. For all of the PITEX analysis, the fault injection and sensed 
fault times are assumed to be the same, therefore, the diagnostic delay time of the sample output in table 2 
is 4.61 seconds. Figure 4 illustrates the changes in diagnostic delay because of the PITEX modifications. 
The average diagnostic time for all the scenarios was 8.21 seconds with a maximum delay of 27.44 
seconds. The decrease in diagnostic time (20 second average delay to approximately 8 seconds) was a 
combined result of the statistically-based monitor algorithm and the modified RTI policies. 
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Figure 4. — Comparison of the Diagnostic Delay Output from Previous 
Version of PITEX and the Current PITEX Software. 

The PITEX testing also included a study of the impact of sensor resolution upon the diagnostic 
output. During this testing phase, simulation data was utilized that contained the sensor resolution 
information for each sensor signal. In all but one case, the diagnostic solution passed the validation 
testing. In the case where the validation testing failed, PITEX had rescinded the correct fault analysis at a 
time over 500 seconds after the original fault indication was made. The root cause was a set of 
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observations that caused PITEX to retract the original fault analysis and indicate no failure had occurred. 
In this case, the insensitivity of several of the sensors involved in the analysis combined with the 
extended failure condition caused the confusion. 

It was found that the impact of the sensor resolution on the diagnostic delay times depended on the 
scenario tested. In several cases, the diagnostic delay increased indicating that parameters utilized in the 
analysis required improved sensor resolutions. By considering the impact of design parameters, such as 
sensor resolution, on the diagnostic analysis, trade studies can be conducted to determine cost of design 
change and increase in diagnostic performance. 

An investigation of the hardware build variation impact on the diagnostic performance of PITEX was 
also conducted. The RP-1 subsystem was used to evaluate the robustness of the diagnostic system to 
specific component variations. Uncertainties were systematically introduced in the simulated data sets by 
varying four key parameters based on information obtained from OSC. 

1 . Setpoint for the primary regulator in the pressurization system (RG1 1) 

2. Opening time for the RP-1 tank feed pneumatic valve (PV07) 

3. Opening time for the RP-1 feed pneumatic valve (PV02) 

4. Opening/closing times for the RP-1 tank primary pressurization valve (SV02) 

Using a Design of Experiment (DOE) methodology, 8 twenty-three test data sets were selected and 
generated, representing the range of nominal variation for these hardware components. Some false alarms 
were detected when these data sets were processed. However, since PITEX was initially developed 
without the hardware information, these false alarms were not unexpected. Adjustments to the threshold 
values in the monitor initialization file and in the diagnostic model corrected the problem. This initial 
study indicated further DOE analysis will need to be conducted over the entire monitored system to 
ensure that hardware variations are accounted for within the diagnostic analysis. 

With the adjusted monitor thresholds, all twenty-four PITEX failure scenarios were reprocessed. With 
one exception, all of the data sets that represent the fault scenarios from table 1 passed the validation 
testing. The one case that did not pass (failure scenario number 21) was a result of the original failure 
scenario not being severe enough. In that failure scenario, the primary pressurization regulator failed its 
high pressure limit. From the hardware uncertainty studies conducted for this component, the allowable 
nominal set point pressure range for this regulator encompassed the original failure value; therefore the 
final PITEX diagnostic solution did not result in the proper detection of this failure. 


B. Computer Resource Utilization 

The second phase of the PITEX testing involved the assessment of the performance of the diagnostic 
software on “flight-like” hardware. Performance metrics evaluated were CPU usage and memory 
utilization. These were selected due to their influence on the real-time operation of the diagnostic system. 

Overall, the CPU usage was very low. The exception to this was when Livingstone performed a 
diagnosis. The RTI requests a diagnosis from Livingstone only when conditions in the system indicate 
that a fault has occurred, so the impact to the CPU usage was minimal since Livingstone is called so 
infrequently. In addition, Livingstone has a low execution priority and therefore, the execution of other 
time critical processes is not inhibited. If CPU usage did reach 100% that would simply result in a longer 
time before a diagnosis was produced. It would not prevent PITEX from detecting a failure. The ability of 
PITEX to perform during periods of limited computer resources was previously demonstrated in earlier 
CPU restriction tests. 4 

For the memory utilization tests, static, stack, and dynamic memory usage were examined. PITEX 
demonstrated that it effectively uses all three. The memory tests demonstrated that PITEX performed well 
within the current memory constraints of the hardware design. The largest consumer of the available 
memory was Livingstone (which also included the RTI for testing purposes). However, as illustrated in 
figure 5, approximately 70% of the memory resources are still available after accounting for the 
diagnostic system. This demonstrated the expansion potential of the PITEX diagnostic system. 
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Figure 5. — Current Memory Usage Chart for PITEX X-34 MPS Application. 


VI. Concluding Remarks 

In general, there are few applications and demonstrations of on-board real-time diagnostics for 
problems that represent the complexity and uncertainty of space transportation or launch systems. To 
address this technology gap, the Propulsion IVHM Technology Experiment (PITEX) developed and 
applied a diagnostic/health assessment system for the X-34 main propulsion system (MPS), a relevant 2nd 
Generation reusable launch vehicle (RLV) subsystem. As reported in this paper, problems relating to the 
lack of experimental data and the diagnostic system’s robustness to system variations were resolved 
during the development of PITEX. By considering flight-like data (noise, sensor resolution, and hardware 
uncertainties), realistic nominal and failure modes, real-time operation, and computer resource 
management issues, PITEX demonstrated that it was a “real” solution for the health assessment of a 
rocket propulsion system. Furthermore, improvements to the Livingstone diagnostic modeling 
environment were demonstrated with the PITEX application. 

Ideally, experimental data would have provided the nominal and failure data required to demonstrate 
and validate the diagnostic system. However due to the unavailability, costs, and risks associated with 
these types of flight tests, that was not possible. Failure data is too costly to gather for rocket propulsion 
systems and unavoidably prohibitive to collect. Therefore, flight-like data were used and the limitations 
associated with a simulated environment were circumvented by including as much real-world information 
as possible in the data. Together, the flight-like data and failure scenarios provided a complete simulation- 
based testing, demonstration, and validation environment for PITEX. Ultimately the certification of health 
assessment tools and systems, particularly for rocket propulsion systems, will require extensive 
simulation-based testing. 

PITEX demonstrated the capability to do reliable diagnostics in the presence of real-world system 
variations and uncertainties by detecting and isolating, at the subsystem level, injected faults correctly. In 
addition, realistic failure modes were utilized based on information obtained from X-34 domain experts at 
Marshall Space Flight Center (MSFC) and Orbital Sciences Corporation (OSC). These failure modes 
were considered critical to the operation of the X-34 MPS and included valves sticking/failing open or 
closed, regulator problems, and sensor and microswitch failures. In addition, the type of testing performed 
on PITEX was unique. Along with the validation of the diagnostic solution, the risks to hardware and 
safety concerns were handled by quantifying resource management needs and performance under CPU 
restrictions. Analysis indicated that the PITEX system used fewer resources than were allocated for the 
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computation of the diagnostic solution, indicating that the diagnostic system could be expanded to cover 
additional components. This in-depth testing advanced the PITEX solution, from a simple laboratory 
environment demonstration to a more relevant environment tested tool. 
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